Papers with nearest neighbor search
Fast Nearest Neighbor Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Fast kNN-MT uses the entire corpus as the datastore for the nearest neighbor search . knn-MT is two-orders slower than vanilla MT models . |
| Approach: | They propose a fast kNN-MT model that uses the entire corpus as the datastore for nearest neighbor search. |
| Outcome: | The proposed model is two-orders faster than kNN-MT and is only two times slower than the standard model. |
Generative Multi-hop Retrieval (2022.emnlp-main)
Copied to clipboard
| Challenge: | A bi-encoder approach to text retrieval has limitations in multi-hop settings; the reformulated query gets longer as the number of hops increases, which further tightens the embedding bottleneck of the query vector. |
| Approach: | They propose an encoder-decoder model that performs multi-hop retrieval by simply generating the entire text sequences of the retrieval targets. |
| Outcome: | The proposed model achieves comparable or higher performance than bi-encoder models in five datasets while demonstrating superior GPU memory and storage footprint. |
ZS-BERT: Towards Zero-Shot Relation Extraction with Attribute Representation Learning (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to relation extraction require labeled data, but labeling is difficult . Existing models cannot recognize rare instances that are never covered by training data . |
| Approach: | They propose a multi-task learning model that directly predicts unseen relations without hand-crafted attribute labeling and multiple pairwise classifications. |
| Outcome: | The proposed model outperforms existing methods by 13.54% on two well-known datasets. |
Zero-Shot Dense Retrieval with Momentum Adversarial Domain Invariant Representations (2022.findings-acl)
Copied to clipboard
| Challenge: | Dense retrieval (DR) methods first encode texts into a dense embedding space and then conduct text retrieval using efficient nearest neighbor search. |
| Approach: | They propose Momentum adversarial Domain Invariant Representation learning to train a domain classifier that distinguishes source versus target domains and adversarially updates the DR encoder to learn domain invariant representations. |
| Outcome: | The proposed method outperforms baselines on 10+ ranking datasets collected in the BEIR benchmark in the zero-shot setting, with more than 10% relative gains on datasets with enough sensitivity for DR models’ evaluation. |
Scalable Zero-shot Entity Linking with Dense Entity Retrieval (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for entity linking use manually curated mention tables and incoming Wikipedia link popularity. |
| Approach: | They propose a BERT-based entity linking model with a bi-encoder that embeds the mention context and the entity descriptions and then re-ranked the candidate with . they also evaluate the accuracy-speed trade-off inherent to large pre-trained models. |
| Outcome: | The proposed model is state-of-the-art on recent zero-shot benchmarks and established non-zero-shot evaluations. |
Large-scale similarity search with Optimal Transport (2023.emnlp-main)
Copied to clipboard
| Challenge: | Word mover's distance (WMD) is a powerful tool for comparing probability distributions in NLP. |
| Approach: | They propose a waterstein distance approximation that uses the L1 embedding method to find the k-nearest neighbors. |
| Outcome: | The proposed approximation performs comparable to the vanilla Wasserstein distance and can be computed three orders of magnitude faster than the vanilla waterstein distance. |
Phrase-BERT: Improved Phrase Embeddings from BERT with an Application to Corpus Exploration (2021.emnlp-main)
Copied to clipboard
| Challenge: | Phrase representations derived from pretrained language models often lack lexical similarity to determine semantic relatedness. |
| Approach: | They propose a contrastive fine-tuning objective that enables BERT to produce more powerful phrase embeddings by fine- tuning a dataset of diverse phrasal paraphrases and a large-scale dataset of phrases in context. |
| Outcome: | The proposed model outperforms baseline models across phrase-level similarity tasks while also showing increased lexical diversity between nearest neighbors in the vector space. |